1. Technical Field
The invention relates to a full-service research bureau and test center. More particularly, the invention relates to a World Wide Web based full-service research bureau and test center.
2. Description of the Prior Art
In the traditional shrinkwrap software world, product usability testing was a business process that was considered nice to have. The software sales cycle relied heavily on product reviews, word-of-mouth, and controlled, static demonstrations (demos). Actual use of a product did not happen until after a user had purchased the product. Costs for customers to switch from a first product to a second product were high, so customer usability was given a low priority.
Today, in the online world, the product is the World Wide Web (Web) site and the Web site is the sales cycle. The usability of a Web site therefore has a direct and critical effect on how much revenue the site can generate. However, usability testing methods have not yet caught up to the requirements of Web site development processes.
Traditional user interface (UI) tests are labor intensive, cost-prohibitive, take ages to run, and yield subjective, non-analytical results at best. Online product competition is now in hyper-drive, and despite multi-million dollar budgets for a typical product launch, Web product managers still do not have an objective, rigorous way of measuring the customer satisfaction and usability of features on their own Web sites. The situation for competitive intelligence is even worse.
The frustration due to the lack of information is summed up by the product manager of a Top 5 Portal Site:
No one in this space knows what they""re doing. The most successful people are also the most superstitious about their methods, since with no hard data, they feel their successxe2x80x94however greatxe2x80x94has only been based on luck and gut feel so far. Everyone has an opinion, but no one can tell me why any one product is really winning or losing.
Poor understanding of a Web usability and feature quality measurement system is due to a number of factors:
1. The pace of technological advance. In the arms race of new technology, companies"" implementation timetables are driven by the competition, not their Information Technology (IT) staff.
2. The multiplicity of functions. Functions for a corporate information Web site are inappropriate for a pre-teen chat room. Designers have to figure out how each function (email, chat, games, shopping, etc.) works best, taking into account the unique audience and objectives of the particular Web site.
3. A flood of raw data but a lack of information. Web logging efforts provide a multitude of traces of user activity. Yet determining what the Web logging data means is impossible because a most important piece, the user""s objective, is missing. The most advanced Web site-level tracking methods available focus on whether a marketer""s objective, i.e. purchase, registration, ad clicks, etc., was met. The user""s objective is ignored. The implication is that a look-to-book ratio twice as good as the industry average might not be cause to celebrate. Perhaps hundreds of willing buyers didn""t purchase an item or service because they didn""t understand the return policy.
4. Little in-depth testing. Traditional user testing methods, such as for example usability laboratories with one-way mirrors, are expensive, labor intensive, and require long turnaround times. In addition, they provide highly subjective results across an insignificant sample size.
O. Davis and V. Jain, Method and Apparatus for Tracking Client Interaction with a Network Resource and Creating Client Profiles and Resource Database, U.S. Pat. No. 5,796,952 (18 Aug. 1998) discloses a method for monitoring client interaction with a resource downloaded from a server in a computer network. The method comprises the steps of using a client to specify an address of a resource located on a first server and downloading a file corresponding to the resource from the first server in response to specification of the address. The method further comprises the steps of using the client to specify an address of a first executable program located on a second server, the address of the first executable program being embedded in the file downloaded from the first server. The first executable program includes a software timer for monitoring the amount of time the client spends interacting with and displaying the file downloaded from the first server. The method further comprises the step of downloading the first executable program from the second server to run on the client so as to determine the amount of time the client interacts with the file downloaded from the first server. The method further comprises the step of using a server to acquire client identifying indicia from the client, and uploading the amount of time determined by the first executable program to a third server. The first executable program may also monitor time, keyboard events, mouse events, and the like, in order to track choices and selections made by a user in the file. It may execute upon the occurrence of a predetermined event, as well as monitor or determine the amount of information downloaded by the client. The monitored information and client identifying indicia is stored on a database in a server for use in analysis and for automatically serving out files assembled according to user interests and preferences.
While an objective of the Davis teachings is to provide means for -creating a database of user profiles containing details of individual user interaction with and use of network resources and of the amount of time spent by users interacting with and/or using particular resources, as well as details of choices created by individual users within a particular resource, Davis does not disclose nor suggests a usability test or pre-qualified testers. Davis does not disclose nor suggest that a customer take part in the design phase of a usability test to accomplish a set of customer objectives, which take into account a user""s objectives.
S. R. Coffey, D. B. Pinsley, and K. A. Poloniewicz, Computer Use Meter and Analyzer, U.S. Pat. No. 5,675,510 (07 Oct. 1997) discloses a system that measures and reports the use of a personal computer by a user through a log file. The log file includes entries corresponding to predetermined events and can report on the applications used and communication functions engaged in by the user. The log files from one or more computers may be assembled and analyzed in order to ascertain computer use habits for computer software, computer hardware and computer communications. The system may also be used to predict computer use trends and to represent computer use history.
The monitoring system of Coffey, et al., provides traces of what users choose, but the disclosure does not teach nor suggest why a users navigate down particular paths. Coffey, et al., discloses that their system collects child Window information for commercial online service providers and user applications. Window titles of these applications"" child Windows generally hold useful descriptions of the activity at that moment. For example, if a subscriber is using a mail system for a service, then the Window title so indicates. The system records those titles in a log file.
Coffey, et al., further discloses that online marketers try to understand the characteristics of Web traffic and how much time users spend at different sites. Traffic statistics become fundamental inputs for media planning, in a fashion analogous to using TV ratings as a basis for buying or pricing commercial time. However, Coffey, et al. does not disclose nor suggest that users"" objectives be considered in any process of the system.
A. N. Goldhaber and G. Fitts, Attention Brokerage, U.S. Pat. No. 5,794,210 (11 Aug. 1998) discloses a system that provides for the immediate payment to computer and other users for paying attention to an advertisement or other negatively priced information distributed over a computer network such as the Internet. It is the business of brokering the buying and selling of the attention of users. A further disclosure allows advertisers to detach their messages from program content and explicitly target their audience. A special icon or other symbol displayed on a computer screen may represent compensation and allow users to choose whether they view an ad or other negatively priced information and receive associated compensation. Targeting users may be provided by reference to a database of digitally stored demographic profiles of potential users. Information is routed to users based on demographics, and software agents can be used to actively seek out users on a digital network. Private profiles are maintained for different users and user information is released to advertisers and other marketers only based on user permission. Users may be compensated for allowing their information to be released. Competing advertisers may bid for the attention of users using automatic electronic systems, e.g. an auction protocol and these concepts can be generalized to provide an electronic trading house where buyers and sellers can actively find each other and negotiate transactions.
While Goldhaber et al. discloses its four basic principles are attention, interest, sponsorship, and privacy, it gains knowledge about the behavior of users through estimates of past behavior. That is, Goldhaber et al. discloses a mechanism by which advertisers actively compete by bidding for a viewer""s attention, wherein the bids are based, in part, on estimates of the viewer""s interest and likelihood to buy. The estimates are derived from access to the viewer""s electronic profiles detailing preferences and past consuming behavior. Goldhaber et al. does not disclose nor suggest a mechanism for learning from a viewer why a viewer chooses particular actions.
Goldhaber et al. discloses demographic profiles constructed through interest questionnaires that a customer completes when subscribing to the disclosed service and also through electronic tracking of the customer""s usage of the service (and other habits). The profiles are dynamic and evolve with the customer""s transaction history. A customer can choose to exclude any transaction (e.g. viewing of certain material or purchasing of certain products) from his profile. The profile is also interactive in that a customer edits the profile at any time to add or delete interest features and to delete any transaction records. Goldhaber et al. does not disclose nor suggest a testing mechanism to measure a customer""s success in achieving the customer""s objectives. Nor does Goldhaber et al. disclose or suggest prompting for context-specific feedback ranging from the aesthetics of the design to the reason a page request is terminated. Furthermore, Goldhaber et al. does not disclose nor suggest aggregating statistics across a testing population and presenting the testing results with recommended actions backed up by analysis.
It would be advantageous to provide a method and apparatus that provides Web product managers with quick and detailed feedback on customer satisfaction of their own and competitive products across a large sample size using efficient techniques.
It would be advantageous to provide a method and apparatus that provides a usability test for typical tasks faced by a customer to a Web product manager""s site. The test measures a customer""s success in achieving the customer""s objectives and also prompts for context-specific feedback ranging from the aesthetics of the Web page design to the reason a page request is terminated.
It would be advantageous to provide a method and apparatus that aggregates statistics across a testing population and presents data with recommended actions backed up by analysis.
The invention herein is a method and apparatus that provides, in a preferred embodiment, Web product managers with quick and detailed feedback on a visitor""s satisfaction of the Web product managers"" own and competitive products. The invention herein in a preferred embodiment provides an end-to-end, automated solution for testing the Usability and Feature Quality(trademark) of any Web site. Unlike other usability and product testing solutions, the invention herein quickly generates highly specific, quantitative results across a large sample size using labor and time efficient techniques.
Specifically, the invention herein aids a customer, which in a preferred embodiment is a Web product manager, in designing a usability test for typical tasks faced by a visitor to the customer""s site. The invention uses proprietary software, which in one embodiment comprises enhancements to a tester""s Web browser, to administer the usability test to a pre-qualified pool of testers meeting desired demographic constraints. The usability tests measure a visitor""s success in achieving the visitor""s objectives and also prompt for context-specific feedback ranging from the aesthetics of the design of the customer""s site to a reason why a page request was terminated. Statistics are aggregated across the testing population and are presented as data with recommended actions backed up by analysis.